Papers with causal methods
Transformer-specific Interpretability (2024.eacl-tutorials)
Copied to clipboard
| Challenge: | Transformers are dominant play-ers in various scientific fields, but their inner workings remain opaque. |
| Approach: | This tutorial presents a trending approach to interpreting Transformers . it uses specific features of the Transformer architecture to quantify context- mixing interactions . |
| Outcome: | This tutorial aims to show how a new trending approach can be applied to Transformer-based models. |
Language Models as Causal Effect Generators (2025.emnlp-main)
Copied to clipboard
| Challenge: | Using sequence-driven structural causal models (SD-SCMs) we characterize how SD-SCAMs enables sampling from observational, interventional, and counterfactual distributions according to the desired causal structure. |
| Approach: | They propose a sequence-driven structural causal model that uses language models to parameterize a structural causal system based on a user-specified DAG. |
| Outcome: | The proposed method outperforms state-of-the-art methods and can underpin auditing of language models for (un)desirable causal effects, such as misinformation or discrimination. |
When Meanings Meet: Investigating the Emergence and Quality of Shared Concept Spaces during Multilingual Language Model Training (2026.eacl-long)
Copied to clipboard
| Challenge: | Recent studies have found that Large Language Models process multilingual inputs in shared concept spaces, thought to support generalization and cross-lingual transfer. |
| Approach: | They investigate the development of language-agnostic concept spaces during pretraining of EuroLLM using the causal interpretability method of activation patching. |
| Outcome: | The proposed model is language-agnostic and enables cross-lingual transfer . the model is able to process multilingual inputs, but lacks cross-linguistic alignment . |